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The  research  conducted  under  Grant  number  AFOSR-88-0180  during  the  period 
16  June  1988  to  30  September  1990  is  partially  documented  in  [2],  [3],  [12],  [13]  and 
the  remainder  of  this  report.  It  is  related  to  previous  work  in  [l],  [5],  [6],  [7],  [8],  [9], 
[10],  [11]  and  [14]. 

Optimization  problems  with  functions  that  are  not  everywhere  differentiable  of¬ 
ten  arise  when  certain  techniques  are  applied  to  large  or  complicated  nonlinear  pro¬ 
gramming  problems  in  order  to  convert  them  to  a  sequence  of  smaller  or  less  complex 
problems.  These  transformation  techniques  include  decomposition,  nested  dissection, 
relaxation,  duality  and/or  exact  Li  penalty  methods  and  often  lead  to  functions  to 
be  minimized  which  are  implicitly  defined.  Being  able  to  solve  such  problems  gives 
an  analyst  flexibility  in  modeling  n  problem  for  solution  and  the  ability  to  exploit 
parallel  processing  in  computation.  Hence,  it  is  important  for  practical  applications 
to  be  able  to  solve  such  problems  and  it  is  the  goal  of  the  project  to  develop  efficient 
solution  methods. 

Recent  joint  work  with  J.-J.  Strodiot  (Namur,  Belgium)  has  produced  practically 
useful  and  theoretically  satisfying  ideas  for  solving  single  variable  minimization  prob¬ 
lems  using  function,  but  not  derivative,  values.  This  effort  has  produced,  what  is 
probably,  the  first  instance  of  a  function-value-only  method  for  nonsmooth  functions 
with  proven  rapid  convergence.  The  corresponding  paper  [13]  is  being  revised  in  or¬ 
der  to  append  some  figures  to  illustrate  various  cases  mentioned  in  the  introduction 
and  considered  in  the  proofs.  The  revision  will  be  submitted  for  publication  as  soon 
as  the  retyping  is  completed.  Our  previous  work  on  this  subject  which  introduced  a 
safeguarded  bracketing  technique  and  developed  some  quadratic  approximation  re¬ 
sults  appeared  in  [14]. 

The  above-mentioned  work  stems  from  a  bracketing  method  using  generalized 
derivatives  at  the  bracket  end  points  developed  and  tested  in  [6],  [8],  [10]  and  [11]. 
Results  from  this  work  and  [7]  already  appear  in  a  textbook  by  K.G.  Murty  [15; 
pp.  410-421].  A  new  result  [12]  in  this  area  is  scheduled  to  appear  in  Vol.  49,  No. 
2  of  Mathematical  Programming.  The  result  states  that  either  the  next  iterate  is 
superlinearly  closer  to  the  solution  than  both  of  the  current  bracket  end  points  or 
the  length  of  the  next  bracket  is  superlinearly  shorter  than  that  of  the  current  bracket. 

Recent  research  has  shown  us  more  specifically  what  needs  to  be  looked  at  care¬ 
fully  in  order  to  develop  an  algorithm  having  better  than  linear  convergence  for 
n-variable  nonsmooth  minimization  problems.  The  problems  we  are  interested  in 
are  those  whose  objectives  have  an  underlying  piecewise  C 2  structure  that  is  not 
explicitly  known.  The  algorithm  we  are  working  on  is  a  2nd  order  bundle  method 
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where  each  iterate  depends  on  a, bundle  of  previous  iterates  and  their  corresponding 
function  and  subgradient  values.  The  2nd  order  nature  has  to  do  with  the  method 
also  employing  a  bundle  of  associated  n  x  n  Hessian  matrix  approximations.  The 
main  idea  for  determining  an  iterate  is  to  solve  a  1st  order  quadratic  programming 
(QP)  subproblem  and  then  to  modify  the  solution  by  a  2nd  order  correction  resulting 
from  the  solution  of  one  linear  system  whose  data  depends  the  constraint  multiplier 
values  from  the  QP  problem  and  on  the  Hessian  estimates  associated  with  positive 
multipliers.  A  better  than  linear  convergence  result  as  in  [9]  based  upon  this  new 
idea  was  presented  in  the  Minisymposium  on  Nonsmooth  Optimization  at  the  April 
1989  SIAM  Conference  on  Numerical  Optimization  held  in  Boston.  One  difficulty 
with  this  approach  is  that  the  2nd  order  system  matrix  may  not  be  positive  definite. 


To  study  this  issue  the  principal  investigator  and  his  Ph.D.  student  research  as¬ 
sistant  have  worked  out  a  method  for  smooth  (one  C2  piece)  minimization  which 
employs  both  BFGS  and  symmetric  rank  one  (SRI)  updated  Hessian  estimates.  The 
BFGS  matrix  is  initialized  and  updated  to  be  positive  definite,  whereas  the  SRI 
matrix  is  initialized  to  be  zero  and  may  not  be  positive  definite  at  other  iterations. 
Powell  [16]  has  shown  that  in  some  situations  the  BFGS  method  can  be  slow  to  ap¬ 
proximate  a  Hessian  accurately,  because  this  accuracy  depends  upon  the  generation 
of  conjugate  directions  which  in  turn  depends  on  exact  line  searches.  The  accuracy 
of  the  SRI  method  only  depends  upon  using  linearly  independent  search  directions 
and,  hence,  the  SRI  matrix  is  used  in  the  second  order  model  of  the  function  in 
our  method.  To  solve  the  resulting  linear  system  with  a  possibly  indefinite  matrix, 
a  truncated  preconditioned  conjugate  gradient  [18]  method  is  employed.  The  pre- 
conditioner  matrix  is  the  positive  definite  BFGS  matrix.  We  can  show  that  if  this 
preconditioned  conjugate  gradient  (PCG)  submethod  is  initialized  with  the  zero  vec¬ 
tor  then  its  1st  subdirection  is  the  ordinary  BFGS  direction  and  every  subiterate  and 
subdirection  generated  is  a  descent  direction  for  the  objective  function  at  its  current 
point.  Hence,  the  PCG  submethod  can  be  terminated  at  any  time  with  a  descent 
direction  which  is  a  modification  of  the  BFGS  direction  designed  to  reduce  *he  value 
of  the  SRI  2nd  order  model.  The  PCG  submethod  must  be  terminated  if  a  subdi¬ 
rection  is  generated  which  is  a  direction  of  nonpositive  curvature  with  respect  to  the 
SRI  matrix.  Using  such  a  search  direction  at  a  point  in  a  region  where  the  objective 
function  is  nonconvex  may  be  beneficial.  Near  the  solution  of  the  main  problem  one 
can  expect  the  BFGS  and  SRI  matrices  to  be  similar  so  the  PCG  algorithm  need 
not  perform  many  subiterations. 

For  an  initial  test  of  the  above  ideas  we  used  the  following  three-variable  version 
of  a  two- variable  function  due  to  Byrd,  Nocedai  and  Yuan[4j: 


□ 
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/(*) 


Distribution/ 


Availability  Codes 


Dl  at 


A«m11  and/or 
-v‘'oial 


where 


A  = 


5.  1.  0 
1.  3.  1. 
0  1.  5. 


and  a  —  0.1. 


The  starting  point  was  xq  =  (cos  70°,  sin  70°,  1.0).  The  BFGS  matrix  was  initialized 
as  a  diagonal  matrix  with  diagonal  elements  (1.,  104, 10.)  and  the  SRI  was  initialized 
as  the  zero  matrix  in  order  to  have  a  neutral  start. 

We  modified  the  code  UNCMIN  due  to  Schnabel,  Koontz  and  Weiss [17]  by  ap¬ 
pending  subroutines  to  update  the  5i?l  matrix  and  perform  the  preconditioned  con¬ 
jugate  gradient  iteration.  We  chose  the  option  in  UNCMIN  which  updates  factors 
of  the  BFGS  matrix.  Also,  we  replaced  the  line  search  subroutine  by  one  written  by 
More  and  Thuente  which  was  changed  slightly  to  impose  only  the  Wolfe  [19]  stopping 
conditions. 

A  user-defined  parameter  l  was  included  to  define  the  maximum  number  of  con¬ 
jugate  gradient  subiterations  allowed  at  each  major  iteration.  When  l  =  0,  our 
algorithm  becomes  a  pure  BFGS  method. 

The  four  runs  summarized  in  the  following  table  had  the  same  first  point  Xi,  as 
well  as  initial  point  x0,  and  were  terminated  with  iterate  xk  when 


max 

\oj\xk) 

<  10"9. 

1 <]<3 | 

dxj  | 

l  =  0 

l  = 

1 

M 

II 

II 

(BFGS) 

iterations 

32 

19 

14  14 

evaluations 

34 

23 

17  20 

SRl-error 

3  x  10~ 

7  1  x  10~7 

6  x  10“5  6  X  10-6 

BFGS-error 

6  x  10“ 

2  2  x  10-3 

6  x  10~2  6  x  10"2 

The  first  row  gives  the  number  of  iterations,  the  second  gives  the  number  of  evalua¬ 
tions  of  /  and  V/,  and  the  third  and  fourth  give  the  terminal  values  of 


max  | (5"  —  7)tJj  and  max  \(B  —  j, 

respectively,  where  the  identity  matrix  7  equals  V2/  at  the  solution  (0,0,0),  5  is 
the  final  SRI  matrix  and  B  is  the  final  BFGS  matrix.  The  difference  in  the  BFGS 
and  SRI  errors  should  be  noted  along  with  the  overall  ability  of  the  algorithm  with 
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/  >  0  to  exploit  the  better  approximation  of  the  SRI  matrix.  The  above  results  were 
presented  at  the  May  1990  TIMS/ORSA  Joint  National  Meeting  held  in  Las  Vegas 
and  in  a  slightly  generalized  form  at  the  November  1990  Pacific  West  Optimization 
Seminar  in  Seattle. 

Finally,  for  solving  the  quadratic  programming  subproblem  in  the  nonsmooth  al¬ 
gorithm  it  may  be  beneficial  to  develop  a  special  purpose  method  based  on  the  Ph.D. 
dissertation  of  A.  Al-Saket  [l].  Two  general  papers  [2]  and  [3l  on  this  subject  have 
been  prepared  for  *  dedication  submission. 
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